Does Central Statistical Monitoring Improve Data Quality? An Analysis of 1,111 Sites in 159 Clinical Trials.

BACKGROUND: Central monitoring aims at improving the quality of clinical research by pro-actively identifying risks and remediating emerging issues in the conduct of a clinical trial that may have an adverse impact on patient safety and/or the reliability of trial results. This paper, focusing on statistical data monitoring (SDM), is the second of a series that attempts to quantify the impact of central monitoring in clinical trials. MATERIAL AND METHODS: Quality improvement was assessed in studies using SDM from a single large central monitoring platform. The analysis focused on a total of 1111 sites that were identified as at-risk by the SDM tests and for which the study teams conducted a follow-up investigation. These sites were taken from 159 studies conducted by 23 different clinical development organizations (including both sponsor companies and contract research organizations). Two quality improvement metrics were assessed for each selected site, one based on a site data inconsistency score (DIS, overall -log10 P-value of the site compared with all other sites) and the other based on the observed metric value associated with each risk signal. RESULTS: The SDM quality metrics showed improvement in 83% (95% CI, 80-85%) of the sites across therapeutic areas and study phases (primarily phases 2 and 3). In contrast, only 56% (95% CI, 41-70%) of sites showed improvement in 2 historical studies that did not use SDM during study conduct. CONCLUSION: The results of this analysis provide clear quantitative evidence supporting the hypothesis that the use of SDM in central monitoring is leading to improved quality in clinical trial conduct and associated data across participating sites.

Training New DMC Members: A Call to Action.

Historically, clinical trialists developed DMC expertise and experience with "on the job training". Clinical trialists have recognized a large and growing gap between the demand and supply of trained DMC members due in part to the huge increase in ongoing clinical trials. A critical need exists to increase the supply of DMC trained clinicians and biostatisticians. Despite a rich collection of published training material, many have recommended a mentoring process to train clinical trialists and data scientists lacking DMC experience. We propose that academic, regulatory, industry and other leaders in clinical trials support including investigators lacking DMC experience as members of a DMC so that they can be trained to learn about the DMC processes during conduct of an actual trial".

Data Monitoring Committees and clinical trials: From scientific justification to organisation.

Clinical trials often last several months or even several years. As the trial progresses, it can be tempting to find out whether the data obtained already answers the question posed at the start of the trial in order to stop inclusions or monitoring earlier. However, knowing and taking into account interim results can sometimes compromise the integrity of the results, which is counterproductive. To minimise this risk and ensure that the treatments are assessed reliably, safety and/or efficacy criteria are monitored during the study by a Data Monitoring Committee. After receiving the results confidentially, the Data Monitoring Committee assesses the benefit/risk ratio of the study treatment and recommends that the trial be continued, modified or terminated. Data Monitoring Committee members issuing these recommendations have an important responsibility: a hasty decision to end the trial may lead to inconclusive results unable to answer the initial question and, inversely, delaying the decision to end the trial may expose the subjects to potentially ineffective or even harmful interventions. The Data Monitoring Committee's task is therefore particularly complex. With this in mind, the round table discussion at the Giens workshops was a chance to review the scientific justification for creating Data Monitoring Committees and to recall the need for their members to receive comprehensive training on the complexities of multiple analyses, confidentiality requirements applying to the results and the need for them to be aware that recommendations to end a trial must be based on data that is robust enough to assess the benefit/risk ratio of the treatment studied.

Guidelines for Data and Safety Monitoring in Pragmatic Randomized Clinical Trials Using Case Studies.

Pragmatic randomized clinical trials (pRCTs) have a unique set of considerations for data and safety monitoring. Because of their unconventional trial designs coupled with collection of multilevel data and implementation outcomes in real-world settings, thoughtful consideration is needed on the presentation of the trial design and accruing data to facilitate review and decision-making by the trial's data and safety monitoring board (DSMB). To our knowledge, there is limited information available in practical guidelines for generalists and medical general practitioners on what to monitor and to report to the DSMB during the conduct of pRCTs and what the DSMB should focus on in its review of reports. This article discusses these matters in the context of 3 case studies focusing on a set of critical data and safety monitoring questions that would be of interest to the generalist conducting pRCTs. In considering these questions, we provide tabular and graphical illustrations of how data can be presented to the DSMB while drawing attention to those areas that the DSMB should focus on in its review of the trial. The strategies and viewpoints discussed herein provide practical guidelines and can serve as a resource for the generalist conducting pRCTs.

Data monitoring committees in pediatric randomized controlled trials registered in ClinicalTrials.gov.

BACKGROUND: Data monitoring committees advise on clinical trial conduct through appraisal of emerging data to ensure participant safety and scientific integrity. While consideration of their use is recommended for trials performed with vulnerable populations, previous research has shown that data monitoring committees are reported infrequently in publications of pediatric randomized controlled trials. We aimed to assess the frequency of reported data monitoring committee adoption in ClinicalTrials.gov registry records and to examine the influence of key trial characteristics. METHODS: We conducted a cross-sectional data analysis of all randomized controlled trials performed exclusively in a pediatric population and registered in ClinicalTrials.gov between 2008 and 2021. We used the Access to Aggregate Content of ClinicalTrials.gov database to retrieve publicly available information on trial characteristics and data on safety results. Abstracted data included reported trial design and conduct parameters, population and intervention characteristics, reasons for prematurely halting, serious adverse events, and mortality outcomes. We performed descriptive analyses on the collected data and explored the influence of clinical, methodological, and operational trial characteristics on the reported adoption of data monitoring committees. RESULTS: We identified 13,928 pediatric randomized controlled trial records, of which 39.7% reported adopting a data monitoring committee, 49.0% reported not adopting a data monitoring committee, and 11.3% did not answer on this item. While the number of registered pediatric trials has been increasing since 2008, we found no clear time trend in the reported adoption of data monitoring committees. Data monitoring committees were more common in multicenter trials (50.6% vs 36.9% for single-center), multinational trials (60.2% vs 38.7% for single-country), National Institutes of Health-funded (60.3% vs 40.1% for industry-funded or 37.5% for other funders), and placebo-controlled (47.6% vs 37.5% for other types of control groups). Data monitoring committees were also more common among trials enrolling younger participants, trials employing blinding techniques, and larger trials. Data monitoring committees were more common in trials with at least one serious adverse event (52.6% vs 38.4% for those without) as well as for trials with reported deaths (70.3% vs 38.9% for trials without reported deaths). In all, 4.9% were listed as halted prematurely, most often due to low accrual rates. Trials with a data monitoring committee were more often halted for reasons related to scientific data than trials without a data monitoring committee (15.7% vs 7.3%). CONCLUSION: According to registry records, the use of data monitoring committees in pediatric randomized controlled trials was more frequent than previously reported in reviews of published trial reports. The use of data monitoring committees varied across key clinical and trial characteristics based on which their use is recommended. Data monitoring committees may still be underutilized in pediatric trials, and reporting of this item could be improved.

Bringing data monitoring committee charters into the sunlight.

Clinical trials investigating novel or high risk interventions, or studying vulnerable participants, often use a data monitoring committee to oversee the progress of the trial. The data monitoring committee serves both an ethical and a scientific function, by protecting the interests of trial participants while ensuring the integrity of the trial results. A data monitoring committee charter, which typically describes the procedures by which data monitoring committees operate, contains details about the data monitoring committee's organizational structure, membership, meeting frequency, sequential monitoring guidelines, and the overall contents of data monitoring committee reports for interim review. These charters, however, are generally not reviewed by outside entities and are rarely publicly available. The result is that a key component of trial oversight remains in the dark. We recommend that ClinicalTrials.gov modify its system to allow uploading of data monitoring committee charters, as is already possible for other important study documents and that clinical trialists take advantage of this opportunity to voluntarily upload the data monitoring committee charter for trials that have one. The resulting cache of publicly available data monitoring committee charters should provide important insights for those interested in a particular trial, as well as for meta-researchers who wish to understand and potentially improve how this important component of trial oversight is actually being applied.

The Data Monitoring Committee: A Collective or a Collection?

In this commentary, we urge that a Data Monitoring Committee (DMC) should operate as a collective, that is, as a unitary whole. In so doing, its recommendations should emerge through a consensus development process, not through a vote of the members. The summary notes of its closed session, that is, its minutes, should report the recommendations of the DMC and, if necessary, the justification for those recommendations; it should not attribute opinions to individual members. Importantly, the proceedings of the DMC meetings should not be electronically recorded.

DMC reports in the 21st century: towards better tools for decision-making.

Data Monitoring Committees (DMCs) have the important task to protect the safety of current and future patients during the conduct of a clinical study. Unfortunately, their work is often made difficult by voluminous DMC reports that are poorly structured and difficult to digest. In this article, we suggest improved solutions. Starting from a principled approach and building upon previous proposals, we offer concrete and easily understood displays, including related computer code. While leveraging modern tools, the most important is that these displays support the DMC's workflow in answering the relevant questions of interest. We hope that the adoption of these proposals can ease the task of DMCs, and importantly, lead to better decision-making for the benefit of patients.

Ensuring the Scientific Value and Feasibility of Clinical Trials: A Qualitative Interview Study.

BACKGROUND: Ethical and scientific principles require that clinical trials address an important question and have the resources needed to complete the study. However, there are no clear standards for review that would ensure that these principles are upheld. METHODS: We conducted semi-structured interviews with a convenience sample of nineteen experts in clinical trial design, conduct, and/or oversight to elucidate current practice and identify areas of need with respect to ensuring the scientific value and feasibility of clinical trials prior to initiation and while ongoing. We used a priori and grounded theory to analyze the data and constant comparative method to induce higher order themes. RESULTS: Interviewees perceived determination of scientific value as the responsibility of the investigator and, secondarily, other parties who review or oversee research. Interviewees reported that ongoing trials are rarely reevaluated due to emerging evidence from external sources, evaluation is complex, and there would be value in the development of standards for monitoring and evaluating evidence systematically. Investigators, IRBs, and/or data monitoring committees (DMCs) could undertake these responsibilities. Feasibility assessments are performed but are typically inadequate; potential solutions are unclear. CONCLUSIONS: There are three domains where current approaches are suboptimal and in which further guidance is needed. First, who has the responsibility for conducting scientific review, whether it be the investigator, IRB, and/or DMC is often unclear. Second, the standards for scientific review (e.g., appropriate search terms, data sources, and analytic plan) should be defined. Third, guidance is needed on the evaluation of ongoing studies in light of potentially new and evolving evidence, with particular reference to evidence from outside the trial itself.

Monitoring Multiple U.S. Government-Supported Covid-19 Vaccine Trials.

Monitoring U.S. Government-Supported Covid-19 Vaccine TrialsOperation Warp Speed was a partnership created to accelerate the development of Covid-19 vaccines. The National Institutes of Health oversaw one data and safety monitoring board to review/monitor all Operation Warp Speed trials. This article describes the challenges faced in monitoring these trials and provides ideas for future similar endeavors.

The Data and Safety Monitoring Board: The Toughest Job in Clinical Trials.

The DSMB: The Toughest Job in Clinical TrialsIn this review article, the authors discuss methods that DSMBs can use to compare the absolute and relative risks of benefits and adverse effects between trial interventions and illustrate how the DSMB can use this approach to evaluate the balance of these competing risks. Two approaches are discussed - the win ratio and the DOOR probability on one treatment relative to another.

Comitês de monitoramento para a proteção de participantes de pesquisa / Monitoring committees for research participant protection / Comités de seguimiento para la protección de los participantes en investigación

Resumo O tema segurança tem sido intensamente discutido, mostrando-se cada vez mais relevante na saúde pública e em projetos de pesquisa envolvendo seres humanos. Participantes de estudos clínicos estão sujeitos a riscos, físicos ou não, que impactam em sua integridade, direitos ou autonomia. Este trabalho apresenta e discute a atuação do Comitê de Monitoramento de Dados e de Segurança para a proteção do participante de pesquisa e minimização de riscos em pesquisa clínica. A metodologia consiste em revisão integrativa da literatura, realizada com o propósito de identificar as funções dos comitês e seu papel na proteção dos participantes. Identificou-se que grande parte das publicações analisadas confirmam que os comitês de monitoramento têm como responsabilidade principal a proteção do participante de pesquisa, além da garantia de integridade e credibilidade da pesquisa.

Abstract Of increasingly relevance in public health and research projects involving human beings, the topic of safety has been intensely discussed. Participants in clinical trials are subject to risks, physical or otherwise, that impact their integrity, rights, or autonomy. This study outlines and discusses the performance of the Data and Safety Monitoring Committee for research participant protection and risk minimization in clinical research. An integrative literature review was conducted to identify the committees' duties and role in protecting participants. Most of the analyzed articles confirm that the monitoring committees are mainly responsible for protecting research participants, as well as ensuring research integrity and credibility.

Resumen La seguridad ha sido un tema muy discutido, por lo que muestra su relevancia para la salud pública y los proyectos de investigación que involucran a seres humanos. Los participantes en estudios clínicos están sujetos a riesgos físicos o de otro tipo, que impactarán su integridad, derechos o autonomía. Este texto realiza un debate sobre el desempeño del Comité de Seguimiento de Datos y Seguridad destinado a la protección de los participantes de investigación y la mitigación de los riesgos en investigación clínica. Se realizó una revisión integradora de la literatura, con el propósito de identificar las funciones de los comités y su rol en la protección de los participantes. La mayoría de las publicaciones analizadas confirman que los comités de seguimiento tienen como principal responsabilidad la protección del participante de la investigación, además de garantizar la integridad y credibilidad de la investigación.

Experiences of the Data Monitoring Committee for the RECOVERY trial, a large-scale adaptive platform randomised trial of treatments for patients hospitalised with COVID-19.

AIM: To inform the oversight of future clinical trials during a pandemic, we summarise the experiences of the Data Monitoring Committee (DMC) for the Randomised Evaluation of COVID therapy trial (RECOVERY), a large-scale randomised adaptive platform clinical trial of treatments for hospitalised patients with COVID-19. METHODS AND FINDINGS: During the first 24 months of the trial (March 2020 to February 2022), the DMC oversaw accumulating data for 14 treatments in adults (plus 10 in children) involving > 45,000 randomised patients. Five trial aspects key for the DMC in performing its role were: a large committee of members, including some with extensive DMC experience and others who had broad clinical expertise; clear strategic planning, communication, and responsiveness by the trial principal investigators; data collection and analysis systems able to cope with phases of very rapid recruitment and link to electronic health records; an ability to work constructively with regulators (and other DMCs) to address emerging concerns without the need to release unblinded mortality results; and the use of videoconferencing systems that enabled national and international members to meet at short notice and from home during the pandemic when physical meetings were impossible. Challenges included that the first four treatments introduced were effectively 'competing' for patients (increasing pressure to make rapid decisions on each one); balancing the global health imperative to report on findings with the need to maintain confidentiality until the results were sufficiently certain to appropriately inform treatment decisions; and reliably assessing safety, especially for newer agents introduced after the initial wave and in the small numbers of pregnant women and children included. We present a series of case vignettes to illustrate some of the issues and the DMC decision-making related to hydroxychloroquine, dexamethasone, casirivimab + imdevimab, and tocilizumab. CONCLUSIONS: RECOVERY's streamlined adaptive platform design, linked to hospital-level and population-level health data, enabled the rapid and reliable assessment of multiple treatments for hospitalised patients with COVID-19. The later introduction of factorial assessments increased the trial's efficiency, without compromising the DMC's ability to assess safety and efficacy. Requests for the release of unblinded primary outcome data to regulators at points when data were not mature required significant efforts in communication with the regulators by the DMC to avoid inappropriate early trial termination.

Evaluating group-sequential non-inferiority clinical trials following interim stopping: The HIV Prevention Trials Network 083 trial.

BACKGROUND/AIMS: The HIV Prevention Trials Network 083 trial was a group-sequential non-inferiority trial designed to compare HIV incidence under a novel experimental regimen for HIV prevention, long-acting injectable cabotegravir, with an active-control regimen of daily oral tenofovir disoproxil fumarate/emtricitabine (brand name Truvada). In March of 2020, just as the trial had completed enrollment, the COVID-19 pandemic threatened to prevent trial participants from attending study visits and obtaining study medication, motivating the study team to update the interim monitoring plan. The Data and Safety Monitoring Board subsequently stopped the trial at the first interim review due to strong early evidence of efficacy. METHODS: Here we describe some unique aspects of the trial's design, monitoring, analysis, and interpretation. We illustrate the importance of computing point estimates, confidence intervals, and p values based on the sampling distribution induced by sequential monitoring. RESULTS: Accurate analysis, decision-making and interpretation of trial results rely on pre-specification of a stopping boundary, including the scale on which the stopping rule will be implemented, the specific test statistics to be calculated, and how the boundary will be adjusted if the available information fraction at interim review is different from planned. After appropriate adjustment for the sampling distribution and overrun, the HIV Prevention Trials Network 083 trial provided strong evidence that the experimental regimen was superior to the active control. CONCLUSIONS: For the HIV Prevention Trials Network 083 trial, the difference between corrected inferential statistics and naive results was quite small-as will often be the case-nevertheless, it is appropriate to report and publish the most accurate and unbiased statistical results.

The challenges of data safety monitoring for a pragmatic study: Lessons from the ADAPTABLE study.

Data monitoring committees (DMCs) play a critical role in protecting the safety of participants and integrity of clinical studies. While there are well-established DMC guidelines for traditional, randomized controlled trials, the clinical trial community is still in the search for best practices in data and safety monitoring in pragmatic clinical trials. ADAPTABLE was a large, open label, pragmatic, randomized controlled trial, harnessing real world data from multiple sources and studying the comparative effectiveness of the two most common dosages of aspirin in patients with atherosclerotic cardiovascular disease. Specific issues arose in ADAPTABLE such as data quality, information latency, and protocol adherence, and these issues were both expected and unexpected features of the pragmatic study design. These issues imposed great challenges to the DMC members who were tasked to make critical decisions during the study. This article summarizes the unique experience of the ADAPTABLE DMC, including the internal debates and concerns, the concerted efforts to accomplish its mission, and the special contribution of the patient representatives. We also offer recommendations on data and safety monitoring for future pragmatic trials.

Data monitoring committee interim reports: We must get there soon!

Currently, too many Data Monitoring Committee Reports for interim review of trial progress are quite inadequate for Data Monitoring Committees to make informed decisions about risks and benefits. Immediate serious improvement is necessary for Data Monitoring Committees to meet their ethical, clinical, and scientific responsibility to trial participants, investigators, sponsors, and participating institutions. To achieve this critical goal, all parties involved in the Data Monitoring Committee process including sponsors, investigators, Data Monitoring Committee members, and the independent statistical reporting group need to have a better understanding of the structure, function, and needs of a Data Monitoring Committee and the content of a Data Monitoring Committee Report. Training modules through the Society for Clinical Trials are now available on their website to facilitate this.